Lazy Bayesian Rules

The naive Bayesian classiier provides a simple and eeective approach to classiier learning, but its attribute independence assumption is often violated in the real world. A number of approaches have sought to alleviate this problem. A Bayesian tree learning algorithm builds a decision tree, and generates a local naive Bayesian classiier at each leaf. The tests leading to a leaf can alleviate attribute inter-dependencies for the local naive Bayesian classiier. However, Bayesian tree learning still suuers from the replication, fragmentation, and small disjunct problems of tree learning. While inferred Bayesian trees demonstrate low average prediction error rates, there is reason to believe that error rates will be higher for those leaves with few training examples. This paper proposes the application of lazy learning techniques to Bayesian tree induction and presents the resulting lazy Bayesian rule learning algorithm, called Lbr. For each test example, it builds a most appropriate rule with a local naive Bayesian classiier as its consequent. It is demonstrated that the computational requirements of Lbr are reasonable in a wide cross-selection of natural domains. Experiments with these domains show that, on average, this new algorithm obtains lower error rates signiicantly more often than the reverse in comparison to a naive Bayesian classiier, C4.5, a Bayesian tree learning algorithm, a constructive Bayesian classiier that eliminates attributes and constructs new attributes using Cartesian products of existing nominal attributes, and a lazy decision tree learning algorithm. It also out-performs, although the result is not statistically signiicant, a selective naive Bayesian classiier.